Multiple frequency band image display system

ABSTRACT

Devices and techniques are generally described for multi-band projection of image data. In various examples, image data may be identified. In some examples, first image data of a first frequency may be generated from the image data. In some examples, second image data of a second frequency may be generated from the image data. In various examples, the first frequency may be higher than the second frequency. In some examples, a first image may be projected onto a projection surface using a raster projector. In an example, the first image may correspond to the first image data. In further examples, a second image may be projected onto the projection surface overlaying the first image. In various other examples, projecting the second image onto the projection surface may be performed using a lamp-based projector. In some examples, the second image may correspond to the second image data.

BACKGROUND

Cameras and other image sensors may be used to capture images and/orvideos of a physical environment. Often, individuals take digitalphotographs of themselves and/or of others to memorialize a moment or toshare photographs on social networks and/or over the internet. Often,images captured using mobile device cameras as well as images found onthe internet are viewed on the relatively small displays that tend to beintegrated into mobile devices. Provided herein are technical solutionsto improve sending of video and other types of data that may reduceproblems associated with changing network conditions.

BRIEF DESCRIPTION OF DRAWINGS

FIG. 1 is a diagram showing an example environment of a display system,arranged in accordance with various aspects of the present disclosure.

FIG. 2 is a diagram showing an example environment with which thedisplay system depicted in FIG. 1 may be used, in accordance withvarious aspects of the present disclosure.

FIG. 3 is a block diagram showing an example architecture of a computingdevice in which the display system described in the present disclosure,or a portion thereof, may be implemented, in accordance with variousembodiments described herein.

FIG. 4 depicts four versions of image content, in accordance with anaspect of the present disclosure.

FIG. 5 depicts a flow chart showing an example process for projectingimage data, in accordance with various aspects of the presentdisclosure.

FIG. 6 depicts a flow chart showing an example process for projectingimage data, in accordance with various aspects of the presentdisclosure.

DETAILED DESCRIPTION

In the following description, reference is made to the accompanyingdrawings that illustrate several examples of the present invention. Itis understood that other examples may be utilized and variousoperational changes may be made without departing from the spirit andscope of the present disclosure. The following detailed description isnot to be taken in a limiting sense, and the scope of the embodiments ofthe present invention is defined only by the claims of the issuedpatent.

Various examples described herein are directed to systems and methodsfor capturing and/or displaying image content. Image data, as describedherein, may refer to stand-alone frames of image data or to multipleframes of sequential image data, appended together to form a video.Image data may be comprised of a plurality of pixels arranged in atwo-dimensional grid including an x component representing a horizontaldirection in the grid and a y component representing a verticaldirection in the grid. A pixel may be the smallest addressable unit ofimage data in an image. A particular pixel may be identified by an xvalue, representing the horizontal position of the pixel in thetwo-dimensional grid and a y value, representing the vertical positionof the pixel in the two-dimensional grid.

FIG. 1 is a diagram showing an example environment 100 of a displaysystem, arranged in accordance with various aspects of the presentdisclosure. The environment 100 comprises image capture device 116, aprojector device 120, and a projection surface 130. Image capture device116 may include, for example, a digital camera module. The digitalcamera module may comprise any suitable type of image sensor device ordevices, such as a charge coupled device (CCD) and/or a complementarymetal-oxide semiconductor (CMOS) sensor effective to capture image datafrom environment 100. Image capture device 116 may include one or morelenses and may be positioned so as to capture images of a portion ofenvironment 100 disposed along an optical axis of image capture device116. In various examples, lenses of image capture device 116 may bewide-angle lenses effective to capture an angle of view greater thanabout 55°. For example, image capture device 116 may be effective tocapture an angle α of a field of view of environment 100. Image capturedevice 116 may include, or may be configured to be in communicationwith, a processing element and/or a memory. Although in FIG. 1 imagecapture device 116 is shown as being angled downward toward a subject110, image capture device 116 may be positioned at any angle withinenvironment 100. Image capture device 116 may include, or may beconfigured to be in communication with, projector device 120.

Projector device 120 may be effective to receive and display image dataand/or video data captured by image capture device 116. In some otherexamples, projector device 120 may be effective to receive and displayimage data received from a computing device. In various examples,projector device 120 may receive image data via a network 104. Forexample, projector device 120 may be effective to receive image datafrom one or more content servers through network 104. Network 104 maybe, for example, the internet, an intranet, a wide area network, a localarea network, or the like. A user of projector device 120 may requestimage data and/or video data from a content server be sent over network104 to projector device 120 for display. For example, a user ofprojector device 120 may control projector device 120 using a companionapplication installed on and executed by a mobile device such as a smartphone or other mobile computing device. The user may request thatcontent from a particular website be displayed by projector device 120through the companion application. The content server may send therequested content to projector device 120 for display on projectionsurface 130. Projector device 120 may be effective to display imageand/or video content sent over network 104 on projection surface 130. Insome examples, projector device 120 may be battery powered or may bepowered by electrically coupling components of projector device 120 toan AC or DC power source.

A user of image capture device 116 may use a companion application tocontrol image capture device 116 to capture images, video, and/or audio(e.g., in instances in which image capture device 116 includes amicrophone) from environment 100. For example, subject 110 may be a userof image capture device 116 and projector device 120. Subject 110 maycontrol image capture device 116 to capture a video of subject 110 andthe portion of the environment 100 that is within the field of view ofthe lens of image capture device 116. Subject 110 may control projectordevice 120 to project the video onto projection surface 130. Subject 110may control playback of the video using video commands such as pause,resume, fast-forward, rewind, slow down, etc. In some further examples,projector device 120 may display a preview image of the field of view ofimage capture device 116. A user of the image capture device 116 mayinstruct image capture device 116 to capture one or more still imageswhile using the projection projected from projector device 120 ontoprojection surface 130 as a preview of the one or more still images. Asis described in further detail below, the user of image capture device116 and/or projector device 120 may control these devices in a varietyof ways. For example, the user (e.g., subject 110) may control imagecapture device 116 and/or projector device 120 using voice commands, acompanion application installed on and executed by a mobile device, adesktop computing device, or other terminal, or by a dedicated hardwarecontroller configured to be in communication with image capture device116 and/or projector device 120. In various examples, the controllerdevice (not shown in FIG. 1) may communicate with image capture device116 and/or projector device 120 using network 104 which may be, forexample, a Wi-Fi network. In some other examples, the controller devicemay communicate with image capture device 116 and/or projector device120 using an infrared signal or other short-range wireless signal (e.g.,a Bluetooth signal), wired communication, or by a user interface ofimage capture device 116 and/or projector device 120.

In some examples, image capture device 116 and projector device 120 maybe included in the same housing and may be configured to be incommunication with one another. In other examples, image capture device116 and projector device 120 may be disposed in separate housings andmay be configured to communicate over a wired or wireless connection. Inexamples where image capture device 116 and projector device 120communicate wirelessly, image capture device 116 and projector device120 may include wireless transmitters and/or receivers (not shown inFIG. 1).

Projector device 120 may be a low-power projector with a short throwratio. The throw ratio of a projector may be a unitless ratio D:Wdefined of the distance D measured from a lens of the projector to theprojection surface 130 to the width of the image W that the projector isto project. In some examples, projector device 120 may have a throwratio of 0.4:1 or greater. Projector device 120 may include one or moretelecentric lenses effective to prevent the edges or borders of adisplayed image from appearing dimmer relative to the center of thedisplayed image. Projector device 120 may be effective to be positionedrelatively near to projection surface 130 (e.g., within 1 meters) with aprojection angle θ between an optical axis of projector device 120 andthe projection surface 130. In some cases, projector device 120 may bepositioned within a few inches (e.g., <10 inches) of projection surface130. Although the projector device 120 depicted in FIG. 1 is angleddownward toward projection surface 130, in other examples, projectordevice 120 may be angled upward toward projection surface 130 and/orlaterally toward projection surface 130, depending on the layout of theparticular environment 100 and the positioning of the projector devicewithin environment 100.

Projector device 120 may include a processor 148 and a memory 150.Memory 150 may be effective to store image data, video data, audio dataand/or executable instructions related to projection of image data,video data, and/or audio data. Processor 148 may be effective to executethe instructions stored in memory 150 and may be configured to controlvarious other components of projector device 120, as described infurther detail below. Although in the examples below, processor 148 isdescribed as performing various actions related to calibration ofprojector device 120 and/or projection of images onto projection surface130; in at least some other examples, the image processing techniquesdescribed below may be performed in whole or in part by one or moreother processing units, including local processing units and/orprocessing units configured to be in communication with projector device120 over network 104. Processing units, as described herein, may includeprogrammable circuits, such as field programmable gate arrays (FPGAs),application specific integrated circuits (ASICs), or processing chipssuch as those included in microcontrollers and/or computing devices.

Projector device 120 may comprise an image filter unit 140, a rasterprojector 142, one or more low frequency projectors 144, one or more IRsensors 146, one or more processors 148, and/or a memory 150. Imagefilter unit 140 may comprise one or more electronic filters effective tofilter out signals above and/or below a particular frequency. Forexample, image filter unit 140 may comprise a high-pass filter effectiveto pass signals with a frequency higher than a particular cutofffrequency. Signals with frequencies below the cutoff frequency may beattenuated by the high-pass filter. Accordingly, a high-pass imagefilter may be effective to filter image data received by projectordevice 120 into frequency components of the image data that are abovethe cutoff frequency. Such frequency components may be referred toherein as “high frequency” components of image data. In some furtherexamples, image filter unit 140 of projector device 120 may comprise alow-pass filter effective to pass signals with a frequency lower than aparticular cutoff frequency. Signals with frequencies above the cutofffrequency may be attenuated by the low-pass filter. Accordingly, alow-pass image filter may be effective to filter image data received byprojector device 120 into frequency components of the image data thatare below the cutoff frequency. Such frequency components may bereferred to herein as “low frequency” components of image data. In somefurther examples, image filter unit 140 of projector device 120 maycomprise a band-pass filter effective to pass frequencies within acertain range and attenuate frequencies outside that range (or “band”).

Image filter unit 140 may comprise one or more of the low-pass,high-pass and/or band-pass filters described above, and may beinstituted as software or may be instituted as one or more dedicatedcircuits in, for example, a digital signal processor (DSP) chip, afield-programmable gate array (FPGA), an application specific integratedcircuit (ASIC) or the like. In various other examples, image filters ofimage filter unit 140 may be instituted as some combination of hardwareand software. In some examples, the highest frequency ranges of imagedata resulting from high-pass filtering may have a frequency of lessthan or equal to 60 line-pairs or pixel-pairs per degree. In somefurther examples, the lowest frequency ranges of image data resultingfrom low-pass filtering may have a frequency of greater than or equal to0.2 line-pairs or pixel-pairs per degree. In some other examples,frequency of filtered image data may be measured in Hertz or otherappropriate units. In addition to filtering based on frequency, in someexamples, image data may be filtered by image filter unit 140 toseparate chrominance values of image data from luminance values of imagedata. In such examples, the chrominance channel and luminance channel ofimage data may be projected separately by projector device 120. Forexample, raster projector 142 may project high frequency luminance datawhile low frequency projectors 144 may project low frequency chrominancedata. In some examples, discrete cosine transform (DCT) may be employedby image filter unit 140 to separate image data into variousfrequencies. In yet other examples, wavelet based image transforms maybe used by image filter unit 140 to separate image data. In someexamples, a portion or region of the image may be identified as a regionof interest. In some examples, one or more regions of interest in theimage data may be projected by a different projector or a differentcombination of projectors relative to other portions of the image datain order to render the region of interest with increased precisionand/or resolution relative to the other portions of the image data.

Projector device 120 may comprise a raster projector 142. Rasterprojector 142 may be a laser projection device and may comprise and/orbe configured in communication with a microelectromechanical (MEMS)scanning mirror. The raster projector 142 may emit laser light that maybe directed onto the MEMS scanning mirror which may, in turn, direct thelight line-by-line in a raster scanning pattern onto projection surface130. Raster projector 142 may not require a lens as laser light emittedby raster projector 142 may be highly collimated. Advantageously, rasterprojector 142 may have a smaller form factor and may consume less powerrelative to traditional lamp-projection systems. Higher color saturationof laser light as compared to non-laser light projected bylamp-projection systems may allow the projector device 120 to projectlower power and lower lumen images that may appear to be equivalent inquality to higher power, higher lumen lamp-projected images.Accordingly, projector device 120 may consume less power thanlamp-projection systems while producing equivalent quality images.

After image data has been separated by a high pass image filter of imagefilter unit 140 into high frequency image data, raster projector 142 maybe effective to project the high frequency image data onto projectionsurface 130. In some examples, processor 148 of projector device 120and/or the MEMS scanning mirror of raster projector 142 may be effectiveto adjust the angle or distance between emitted raster scan lines in anon-uniform fashion based on the distance between the raster projector142 and the position on projection surface 130 where the raster scanline is to be projected, in order to maintain the quality of theprojected image.

For example, the angle or distance between raster scan lines may bedecreased at the point of emission from projector device 120 as the scanlines are projected further away from the raster projector 142 onprojection surface 130. The angle or distance between raster scan linesmay be adjusted by movements of the MEMS scanning mirror of rasterprojector 142 such that light incident on the MEMS scanning mirror isreflected toward a particular position on projection surface 130. Forexample, the MEMS scanning mirror may be effective to cause laser lightemitted from the raster projector 142 to be reflected from the MEMSscanning mirror at different angles. In the example depicted in FIG. 1,the raster scan lines projected at the top of the projection surface 130would be projected with the shortest distance or smallest angle betweenprojected raster scan lines at the point of emission from projectordevice 120. As projection surface 130 is traversed in a downwarddirection (relative to the standing subject 110 in FIG. 1), the distanceor angle between projected raster scan lines will be decreased byprocessor 148 in order to maintain image quality. Similarly, if theprojector device 120 was disposed below the projection surface 130, thedistance or angle between raster scan lines would be decreased whiletraversing the projection surface 130 in an upward direction. Ingeneral, the further the portion of the projection surface 130 is fromthe raster light source, the smaller the distance between raster scanlines projected by raster projector 142. Although, the steps betweenraster scan lines may be reduced by processor and/or MEMS scanningmirror at the point of emission from projector device 120, the scanlines appearing on projection surface 130 may be separated by a uniformdistance. This is because the processor and/or MEMS scanning mirroraccount for the increased divergence of light beams that travel furtherfrom the light emission source by dynamically adjusting the anglebetween raster scan lines at the point of emission from projector device120, as described above.

In various examples, projector device 120 may comprise one or moreinfrared (IR) sensors 146. Infrared sensors 146 may be used to detectinfrared light reflected from projection surface 130 in order todetermine information about projection surface 130 and/or generate adepth map and/or height map of projection surface 130. In some examples,non-infrared depth sensors, such as passive stereo camera pairs, ornon-identical camera pairs, may be used in projector device 120 in placeof, or in addition to, infrared sensors 146. Such non-infrared depthsensors may be used to determine information about projection surface130 and/or generate a depth map and/or height map of projection surface130. Additionally, in some examples, passive stereo camera pairs may useambient light of an environment in which projector device 120 issituated to generate a depth map. Stereo camera pairs may locate thedisplacement (sometimes referred to as “parallax” or “disparity”) ofobjects and/or features between the left image and the right imagecaptured by the stereo camera pair.

In various examples, a depth map may model the surface shape,reflectance, color, and microgeometry of the projection surface 130.Similarly, a height map may model the projection surface 130 using theprojection surface itself as a reference. In some examples, a mesh ofthe projection surface 130 may be computed based on either the depth mapor the height map of projection surface 130. Raster projector 142 may beeffective to emit infrared light used to model projection surface 130and to determine a distance between projector device 120 and variousportions of projection surface 130. In some other examples, projectordevice 120 may include a separate infrared light source apart fromraster projector 142.

As described above, when projecting images and/or videos onto projectionsurface 130, processor 148 and/or a scanning MEMS mirror of rasterprojector 142 may decrease the projection angle of scan lines as thescan lines are projected further and further away from the rasterprojector 142. Projector device 120 may be effective to determine thedistance that individual raster scan lines are to be projected based onthe raster scan line's position within the image data and based on thedepth map generated using data sensed by the one or more IR sensors 146(e.g., by using time-of-flight data).

In various examples, projector device 120 may emit infrared light withraster projector 142 only during a calibration process. The calibrationprocess may be performed during startup of the projector device or whenambient conditions have changed between uses. For example, if theprojection surface 130 changes, the calibration procedure may beinstituted. In other examples the calibration procedure may beinstituted based on a local or remote command to calibrate orrecalibrate the projector device 120. In various further examples, theprojector device 120 may be calibrated upon powering on or “waking” froma low power state.

In some examples, projector device 120 may include a camera 152 with animage sensor facing toward the projection surface 130 and used to senseambient light and color of reflected light from the projection surface130. In various examples, the processor 148 may be effective to controlthe raster projector 142 and/or one or more low frequency projectors 144to adjust colors of light projected by projector device 120 based oncolors of light reflected from projection surface 130 and sensed by thecamera 152. In an example, processor 148 may compare a color value oflight reflected from a portion of the image projected on projectionsurface 130 to a color value of the corresponding portion of the imagestored in memory 150. Processor 148 may be effective to instructprojector device 120 to adjust the color value of light based on adiscrepancy between the reflected color value detected by the imagesensor facing projection surface 130 and the expected color value of theimage data stored in memory 150.

For example, projection surface 130 may be a white wall with ahorizontal red stripe across the white wall. Camera 152 may detect thecolor information of the projection surface 130. Processor 148 may alterthe color of the image projected by raster projector 142 and/or by oneor more low frequency projectors 144 to correct for the coloredprojection surface 130, such that colors of the projected image appearas though projected on a uniformly-colored projection background (e.g.,a white matte background). Similarly, camera 152 may detect ambientlight information from environment 100. Processor 148 may alterluminance values of the image projected by raster projector 142 and/orby one or more low frequency projectors 144 to correct for ambient lightinformation detected by camera 152.

In some examples, light from an image projected by projector device 120may be detected by camera 152 as the light projected by projector device120 is reflected from projection surface 130. Processor 148 may comparecolor values of the projected image detected by camera 152 to colorvalues of the image data corresponding to the projected image stored inmemory 150. If processor 148 determines that a discrepancy existsbetween the projected color values and the color values stored in memory150, processor 148 may execute a color correction algorithm to determineadjustments to the color values of the projected image to correct thediscrepancy. Similarly, processor 148 may correct and/or adjust fordiscrepancies in apparent geometry of the projected image caused bynon-uniform elements in projection surface 130 by comparing theprojected image data to the image data stored in memory 150.

Projector device 120 may further comprise one or more low frequencyprojectors 144. Low frequency projectors 144 may be lamp-basedprojectors, such as an LED projector, and may be effective to projectlight representing image data through one or more telecentric lens ontoprojection surface 130. In at least some examples, each low frequencyprojector 144 may be used to project a particular frequency band ofimage data produced after the image data has been separated intodifferent frequency bands by image filters of image filter unit 140. Forexample, projector device 120 may comprise raster projector 142, a firstlow-frequency projector 144 and a second low-frequency projector 144.Image filters of image filter unit 140 may comprise one or more of ahigh-pass filter, a low-pass filter, and a band-pass filter. As such,image data may be separated into a high-frequency band, a mid-frequencyband, and a low-frequency band. Raster projector 142 may project thehigh-frequency band of the image data onto projection surface 130. Thefirst low-frequency projector 144 may project the mid-frequency band ofthe image data resulting from the band-pass filter onto projectionsurface 130 through a telecentric lens. Similarly, the secondlow-frequency projector 144 may project the low-frequency band of theimage data resulting from the low-pass filter onto projection surface130 through a different telecentric lens. Each telecentric lens may bearranged so as to have the same field of view. The field of view of eachtelecentric lens is a function of the focal length of the telecentriclens and a size of the imager of the particular low-frequency projector144. Although referred to herein as “low-frequency” projectors 144, suchprojectors may be effective to project any frequency band of image dataand may be referred to herein as “low-frequency” simply to reflect thefact that it may be advantageous, in some embodiments, to use thelamp-based projectors to project lower frequency bands of image data,relative to the high frequency image data projected by raster projector142, in order to conserve power usage.

In another example, a single low frequency projector 144 may be used toproject a frequency band of image data produced after the image data hasbeen separated into a high-frequency component and a low-frequencycomponent by image filters of image filter unit 140. For example,projector device 120 may comprise raster projector 142 and a singlelow-frequency projector 144. Image filter unit 140 may comprise ahigh-pass filter effective to pass all image data with a frequency valuegreater than or equal to x and a low-pass filter effective to pass allimage data with a frequency less than x. As such, image data may beseparated into two frequency bands: a high-frequency band and alow-frequency band. Raster projector 142 may project the high-frequencyband of the image data onto projection surface 130. The low-frequencyprojector 144 may project the low-frequency band of the image dataresulting from the low-pass filter onto projection surface 130 through atelecentric lens.

Projector device 120 may undergo an alignment calibration to alignoverlapping frequency bands of projected images such that the resultantimage projected on projection surface 130 is aligned and does not appearblurry. Such a calibration may be performed by detecting the projectedfrequency bands with camera 152 and comparing such image data to imagedata stored in memory 150. In some examples, special calibration imagesmay be used to perform the alignment calibration. In various examples,the alignment calibration may be performed when the projector device 120is produced in a manufacturing setting or refurbishing setting. Invarious other examples, a user of projector device 120 may institute analignment calibration through a user interface of projector device 120or through an interface effective to control projector device 120.

Although in FIG. 1 projector device 120 is shown in environment 100along with image capture device 116, in at least some examples,projector device 120 may function independently of image capture device116 and may not require input from image capture device 116. Forexample, as described above, projector device may receive image datafrom a content server over network 104. In another example, a user mayupload or download image data to memory 150 and may control projectordevice 120 to project image data stored in memory 150.

FIG. 2 is a diagram showing one example of an environment 200 with whichthe display system depicted in FIG. 1 may be used, in accordance withvarious aspects of the present disclosure. In the example shown in FIG.2, projector device 120 comprises image capture device 116, although, asdescribed above, in other examples, image capture device 116 may beseparate from and function independently from projector device 120.

The environment 200 comprises projector device 120 and users 204 a, 204b, 204 c, 204 n. Each user 204 a, 204 b, 204 c, and 204 n may use one ormore user devices such as, for example, mobile device 206, tabletcomputer 208, laptop computer 210, and/or display device 212. Althoughfour users 204 a, 204 b, 204 c, 204 n are shown, any suitable number ofusers may be part of the environment 200. Also, although each user 204a, 204 b, 204 c, 204 n shown in FIG. 2 is associated with a particulardevice (such as mobile device 206 associated with user 204 a, tabletcomputer 208 associated with user 204 b, display device 212 associatedwith user 204 c, and laptop computer 210 associated with user 204 n),each user 204 a, 204 b, 204 c, 204 n may use additional user devices orfewer user devices from what is shown. Additionally, each user 204 a,204 b, 204 c, 204 n may use different user devices apart from what isshown in environment 200 of FIG. 2.

Projector device 120 may perform the various utilities described hereinincluding, for example, short throw projection of images and videos ontoa projection surface 130 (depicted in FIG. 1). As shown and described,projector device 120 may comprise one or more image capture devices 116,one or more cameras 152, one or more processors 148, and/or one or morememories 150. Although not depicted in FIG. 2, projector device may alsocomprise one or more infrared sensors, one or more image filters, one ormore raster projectors, and/or one or more lamp-based projectors, asdescribed above with respect to FIG. 1. In some examples, the memory 150may store images captured by the one or more camera modules 116, orreceived from the various user devices, as well as instructions forperforming image perspective transformation. The various components 150,116, 148 of the projector device 120 may be at a common geographiclocation and/or may be distributed across multiple geographic locations.For example, the projector device 120 and image processing (e.g., colorcorrection and image geometry correction) associated therewith may beimplemented in whole or in part as a cloud or Softare as a Service(SaaS) system. In some examples, the projector device 120 may projectand/or perform color correction, alignment calibration, geometriccorrection, etc. on images received from multiple different users 204 a,204 b, 204 c, 204 n (e.g., via their associated cameras, computingdevices, or other devices). In various other examples, projector device120 may capture images using one or more image capture devices 116.Various user devices (such as mobile device 206 associated with user 204a, tablet computer 208 associated with user 204 b, display device 212associated with user 204 c, and laptop computer 210 associated with user204 n) may include a companion application effective to sendinstructions to image capture device 116 and/or projector device 120.For example, user 204 a may execute a companion application on mobiledevice 206 and may send commands to image capture device 116 and/orprojector device 120. In various examples, user 204 a may use thecompanion application to capture image data with image capture device116 and to project the captured image data with projector device 120. Insome further examples, user 204 a may instruct projector device 120 toenter a calibration mode in order to perform a color correction and/orimage geometry correction to images projected by projector device 120.

The various components of the environment 200 may be in communicationwith one another via a network 104. As described previously, the network104 may be and/or comprise any suitable wired or wireless networkconfigured according to any suitable architecture or protocol. In someexamples, the network 104 may comprise the Internet.

User devices, such as mobile device 206, tablet computer 208, displaydevice 212, and laptop computer 210 may be utilized to control imagecapture device 116 to capture still and/or video images. Similarly, userdevices, such as mobile device 206, tablet computer 208, display device212, and laptop computer 210 may be utilized to control projector device120 to project still and/or video images. In various examples, userdevices may execute a companion application to control operation ofimage capture device 116 and/or projector device 120.

In some examples, user devices including mobile device 206, tabletcomputer 208, display device 212, and/or laptop computer 210 may beconfigured to communicate with other components of the environment 200utilizing, for example, a wired or wireless connection. For example,mobile device 206, tablet computer 208, display device 212, and/orlaptop computer 210 may send and receive data (such as, for example,commands and/or image data) via a wired connection, such as UniversalSerial Bus (USB), or wireless connection, such as near fieldcommunication (NFC) or Bluetooth. In some examples, the user devices maybe configured to receive still images and/or video directly from imagecapture device 116, for example, via the network 104. Although userdevices are described as mobile device 206, tablet computer 208, displaydevice 212, and/or laptop computer 210, the user devices may be anysuitable type of computing device comprising at least one processor andnon-transitory computer-readable memory. In some examples, the userdevices may be configured to receive image frames captured by the imagecapture device 116 and projected by projector device 120. Also, in someexamples, the user devices may comprise one or more camera modules andassociated optics for capturing images and/or video and uploading theresulting frames to projector device 120 for display. In some examples,the user devices, such as mobile device 206, tablet computer 208,display device 212, and/or laptop computer 210, may be configured tocommunicate on a cellular or other telephone network.

In various examples, users, such as users 204 a, 204 b, 204 c, 204 maycontrol projector device 120 and/or image capture device 116 usingaudible commands. For example, a user 204 a may speak a “wake word” thatmay be a spoken, audible command. A wake word may be, for example, aword or phrase for which a wake word engine of image capture device 116and/or projector device 120 continually listens. A microphone of imagecapture device 116 and/or projector device 120 may detect the spokenwake word and, in response, subsequent audio captured by the microphonewill be processed to detect further audible commands and/or thesubsequent audio received by the microphone of image capture device 116and/or projector device 120 may be transmitted to a voice recognitionserver 220. In the example, user 204 a may “wake” the image capturedevice 116 and/or projector device 120 to further voice commands usingthe wake word, and may thereafter speak an audible command for imagecapture device 116 to take a video or take a picture. Similarly, a usermay speak an audible command for projector device 120 to project aparticular image or video, or to enter a calibration mode. Audio may betransmitted/streamed from projector device 120 and/or image capturedevice 116 over network 104 to voice recognition server 220 in any audiofile format, such as mp3, mp4, or the like. Voice recognition server 220may receive the transmitted or streamed audio. Upon determining that theaudio content has reached an endpoint, voice recognition server 220 mayanalyze the received audio stream and may translate the audio streaminto natural language. Voice recognition server 220 may determinewhether or not the natural language corresponds to a command. If so, thevoice recognition server 220 may send the command over network 104 toimage capture device 116 and/or projector device 120. For example, auser 204 a may speak the command, “Take a picture” to projector device120 and/or image capture device 116. Projector device and/or imagecapture device 116 may transmit the voice command to voice recognitionserver 220. Voice recognition server 220 may analyze the audio streamand may translate the audio stream into natural language. Voicerecognition server 220 may determine that the natural language “Take apicture” corresponds to a command effective to instruct projector device120 to capture an image using image capture device 116. Voicerecognition server 220 may send the command over network 104 toprojector device 120. The command may be effective to cause imagecapture device 116 to capture an image.

In other examples, a user 204 a may speak the command, “Project BirthdayVideo” to projector device 120. Projector device 120 may transmit thevoice command to voice recognition server 220. Voice recognition server220 may analyze the audio stream and may translate the audio stream intonatural language. Voice recognition server 220 may determine that thenatural language “Project Birthday Video” corresponds to a commandeffective to instruct projector device 120 to project a video titled“Birthday Video” onto a projection surface. In various examples,“Birthday Video” may be stored in memory 150 or may be stored in adifferent memory accessible by projector device 120 over network 104.Voice recognition server 220 may send the command over network 104 toprojector device 120. The command may be effective to cause projectordevice 120 to access and project the appropriate video.

In some embodiments, the microphone for capturing voice commands may beprovided on a different device separate from the projector device 120and the image capture device 116. The processing of the voice commandand/or transmission of the audio to the voice recognition server 220 maysimilarly be performed by a device other than the image capture device116 and the projector device 120.

In various examples in which color correction, depth map creation, imagecalibration, addition of augmented reality image data (e.g, “AR skins”)is implemented at least in part in a cloud service or SaaS environment,such techniques may be performed at an image transformation device 230.Although depicted as different computing devices in FIG. 2, in someexamples, image transformation device 230 and voice recognition server220 may be implemented in the same housing. Similarly, in variousexamples, image transformation device 230 may be implemented in the samehousing as projector device 120. In yet other examples, imagetransformation device 230 may receive image data captured by imagecapture device 116 and/or camera 152 via network 104. After performingimage transformation in accordance with the various techniques describedherein, image transformation device 230 may send transformed image dataover network 104 to projector device 120 and/or to one or more userdevices and/or other computing devices, such as, for example, a socialmedia server. In some examples, transformed image data may be sent to acomputer vision system (not shown). The computer vision system may beprogrammed to recognize various features of a subject or subjectsdepicted in the perspective-transformed images. For example, thecomputer vision system may be programmed to recognize a face of asubject. In some other examples, the computer vision system may beprogrammed to recognize articles of clothing worn by a subject. Clothesmay be identified by matching a particular item being worn by a subjectto a particular item of clothing known to have been purchased by theuser of image capture device 116 and/or projector device 120 or storedin a database, such as an online-shopping catalog database. For example,the computer vision system may be in communication with one or moreother computing systems that include profile information related to thesubject. The computer vision system may identify particular articles ofclothing worn by a subject by querying other computer systems, such as aserver of an online-shopping website from which the user has purchasedthose articles of clothing. Similarly, the computer vision system mayidentify a subject by querying a computer system hosting a social mediaplatform, which can provide to the computer vision system informationabout the subject (e.g., information about clothing purchased by thesubject, worn by the subject in photos available to the social mediaplatform, or other types of information available to social mediaplatforms) to assist with the identification of that clothing by thecomputer vision system. In various examples, the computer vision systemmay be effective to insert metadata into the perspective-transformedimage. In some examples, such metadata may be optionally displayed whenimage data is projected by projector device 120. The metadata maycomprise a metadata “tag,” or a hyperlink that, which selected by theuser, will direct the user to a retail website where the particulararticle of clothing can be purchased.

FIG. 3 is a block diagram showing an example architecture 300 of a userdevice, such as the projector devices, image capture devices, cameras,display devices, mobile devices, and other computing devices describedherein. It will be appreciated that not all user devices will includeall of the components of the architecture 300 and some user devices mayinclude additional components not shown in the architecture 300. Thearchitecture 300 may include one or more processing elements 304 forexecuting instructions and retrieving data stored in a storage element302. The processing element 304 may comprise at least one processor. Anysuitable processor or processors may be used. For example, theprocessing element 304 may comprise one or more digital signalprocessors (DSPs). In some examples, the processing element 304 may beeffective to filter image data into different frequency bands, asdescribed above. The storage element 302 can include one or moredifferent types of memory, data storage, or computer-readable storagemedia devoted to different purposes within the architecture 300. Forexample, the storage element 302 may comprise flash memory,random-access memory, disk-based storage, etc. Different portions of thestorage element 302, for example, may be used for program instructionsfor execution by the processing element 304, storage of images or otherdigital works, and/or a removable storage for transferring data to otherdevices, etc.

The storage element 302 may also store software for execution by theprocessing element 304. An operating system 322 may provide the userwith an interface for operating the user device and may facilitatecommunications and commands between applications executing on thearchitecture 300 and various hardware thereof. A transfer application324 may be configured to receive images and/or video from another device(e.g., a mobile device, image capture device, and/or display device) orfrom an image sensor 332 included in the architecture 300 (e.g., imagecapture device 116 and/or camera 152). In some examples, the transferapplication 324 may also be configured to upload the received images toanother device that may perform processing as described herein (e.g., amobile device, another computing device, and/or transformation device230). In some examples, a calibration application 326 may performprocessing on image data stored and/or projected by a projector device120 of the architecture 300 and/or from another device. For example,calibration application 326 may perform color correction to account fordiscrepancies between colors sensed from a projection surface and colorsassociated with image data stored in memory. Similarly, calibrationapplication 326 may perform image geometry correction to account forprojection of images on non-flat surfaces. Further, calibrationapplication 326 may be effective to perform an alignment calibration foraligning multiple projections systems of projector device 120. Forexample, calibration application 326 may be effective to alignoverlapping frequency bands of projected images such that the resultantimage projected on projection surface 130 (depicted in FIG. 1) isaligned and does not appear blurry. The alignment calibration may beperformed by detecting the projected frequency bands with camera 152(depicted in FIG. 2) and comparing such image data to image data storedin memory 150 (depicted in FIG. 1). In some examples, specialcalibration images may be used by calibration application 326 to performthe alignment calibration. In various examples, the alignmentcalibration may be performed when the projector device 120 is producedin a manufacturing setting or refurbishing setting. In various otherexamples, a user of projector device 120 may institute an alignmentcalibration through a user interface of calibration application 326.

In some examples, storage element 302 may include a raster scanadjustment utility 350. The raster scan adjustment utility 350 may beconfigured to adjust the distance or angle between raster scan lines atthe point of projection based on the distance between the projectionlight source and the surface onto which the raster scan line is to beprojected. The raster scan adjustment utility may allow the rasterprojector 142 (depicted in FIG. 1) to project a non-distorted image evenwhen projector device 120 is placed a short distance (e.g., ≤3 meters)from projection surface 130. As described above, the raster scanadjustment utility 350 may account for the increased divergence ofraster scan lines as the raster scan lines are projected further andfurther away from the raster projector 142. Raster scan adjustmentutility 350 may determine the distance that raster scan lines areprojected using infrared light projected by raster projector 142 andreflected from projection surface 130. Infrared sensors 146 may detectthe reflected infrared light. Processor 148 and/or raster scanadjustment utility 350 may determine a distance of the projected rasterscan lines based on the time-of-flight of the projected and reflectedinfrared light.

When implemented in some user devices, the architecture 300 may alsocomprise a display component 306. The display component 206 may compriseone or more light-emitting diodes (LEDs) or other suitable displaylamps. Also, in some examples, the display component 206 may comprise,for example, one or more devices such as cathode ray tubes (CRTs),liquid-crystal display (LCD) screens, gas plasma-based flat paneldisplays, LCD projectors, raster projectors, infrared projectors orother types of display devices, etc. In various examples, the displaycomponent 206 may be effective to show preview images and/or thumbnailsof the images and/or videos to be projected by projector device 120.

The architecture 300 may also include one or more input devices 308operable to receive inputs from a user. The input devices 308 caninclude, for example, a push button, touch pad, touch screen, wheel,joystick, keyboard, mouse, trackball, keypad, light gun, gamecontroller, or any other such device or element whereby a user canprovide inputs to the architecture 300. These input devices 308 may beincorporated into the architecture 300 or operably coupled to thearchitecture 300 via wired or wireless interface. In some examples,architecture 300 may include a microphone 370 for capturing sounds, suchas voice commands. Voice recognition engine 380 may interpret audiosignals of sound captured by microphone 370. In some examples, voicerecognition engine 380 may listen for a “wake word” to be received bymicrophone 370. Upon receipt of the wake word, voice recognition engine380 may stream audio to a voice recognition server for analysis, asdescribed above in reference to FIG. 2. In various examples, voicerecognition engine 380 may stream audio to external computing devicesvia communication interface 312.

When the display component 306 includes a touch-sensitive display, theinput devices 308 can include a touch sensor that operates inconjunction with the display component 306 to permit users to interactwith the image displayed by the display component 306 using touch inputs(e.g., with a finger or stylus). The architecture 300 may also include apower supply 314, such as a wired alternating current (AC) converter, arechargeable battery operable to be recharged through conventionalplug-in approaches, or through other approaches such as capacitive orinductive charging.

The communication interface 312 may comprise one or more wired orwireless components operable to communicate with one or more other userdevices such as the user devices depicted in FIG. 2 (including mobiledevice 206 associated with user 204 a, tablet computer 208 associatedwith user 204 b, display device 212 associated with user 204 c, andlaptop computer 210 associated with user 204 n). For example, thecommunication interface 312 may comprise a wireless communication module336 configured to communicate on a network, such as the network 104,according to any suitable wireless protocol, such as IEEE 802.11 oranother suitable wireless local area network (WLAN) protocol. A shortrange interface 334 may be configured to communicate using one or moreshort range wireless protocols such as, for example, near fieldcommunications (NFC), Bluetooth, Bluetooth LE, etc. A mobile interface340 may be configured to communicate utilizing a cellular or othermobile protocol. A Global Positioning System (GPS) interface 338 may bein communication with one or more earth-orbiting satellites or othersuitable position-determining systems to identify a position of thearchitecture 300. A wired communication module 342 may be configured tocommunicate according to the USB protocol or any other suitableprotocol. In various examples where architecture 300 representsprojector device 120 and/or image capture device 116 (shown in FIG. 1),mobile interface 340 may allow projector device 120 and/or image capturedevice 116 to communicate with one or more other computing devices suchas the various computing devices shown in FIG. 2. For example, projectordevice 120 and/or image capture device 116 may receive a command from auser device, an application of a user device, or a voice recognitionserver to capture and/or project an image. Projector device 120 and/orimage capture device 116 may receive a command from the user device tosend the captured image frame to the mobile device or to a social mediasite. In another example, a user device may be effective to send aninstruction to a social media website or other content server to haveimage data sent to projector device 120 for display on projectionsurface 130.

The architecture 300 may also include one or more sensors 330 such as,for example, one or more position sensors, image sensors, and/or motionsensors. An image sensor 332 is shown in FIG. 3. Some examples of thearchitecture 300 may include multiple image sensors 332. For example, apanoramic camera system may comprise multiple image sensors 332resulting in multiple images and/or video frames that may be stitchedand may be blended to form a seamless panoramic output. An example of animage sensor 332 may be camera 152 shown and described in FIGS. 1 and 2.As described, camera 152 may be configured to capture color information,image geometry information, and/or ambient light information related toprojection of image data onto a projection surface 130.

Motion sensors may include any sensors that sense motion of thearchitecture including, for example, gyro sensors 344 and accelerometers346. Motion sensors, in some examples, may be used to determine anorientation, such as a pitch angle and/or a roll angle, of image capturedevice 116 and/or projector device 120 (shown in FIG. 1). The gyrosensor 344 may be configured to generate a signal indicating rotationalmotion and/or changes in orientation of the architecture (e.g., amagnitude and/or direction of the motion or change in orientation). Anysuitable gyro sensor may be used including, for example, ring lasergyros, fiber-optic gyros, fluid gyros, vibration gyros, etc. Theaccelerometer 346 may generate a signal indicating an acceleration(e.g., a magnitude and/or direction of acceleration). Any suitableaccelerometer may be used including, for example, a piezoresistiveaccelerometer, a capacitive accelerometer, etc. In some examples, theGPS interface 338 may be utilized as a motion sensor. For example,changes in the position of the architecture 300, as determined by theGPS interface 338, may indicate the motion of the GPS interface 338.Infrared sensor 360 may be effective to determine a distance between asurface, such as projection surface 130 and projector device 120 (shownin FIG. 1). In some examples, the infrared sensor 146 may determine thecontours of the surface and may be capable of using computer visiontechniques to recognize facial patterns or other markers within thefield of view of the infrared sensor 146's camera. In some examples, theinfrared sensor 146 may include an infrared projector and camera. Inother examples, a raster projector of architecture 300 (e.g., rasterprojector 142 depicted in FIG. 1) may project infrared light that may besensed by infrared sensor 146. Processing element 304 may build a depthmap based on detection by the infrared camera of a pattern of structuredlight displayed on a surface by the infrared projector. In some otherexamples, the infrared sensor 146 may include a time of flight camerathat may compute distance based on the speed of light by measuring thetime of flight of a light signal between a camera of the infrared sensor146 and a surface, such as subject 110 or projection surface 130 shownin FIG. 1. In various examples, infrared sensor 146 may be effective todetermine the pitch angle and/or roll angle of projector device 120and/or image capture device 116. Further, in some examples, processingelement 304 may be effective to determine the location of variousobjects in the physical environment within the field of view of imagecapture device 116 and/or projector device 120 based on the depth mapcreated by the infrared sensor 146. As noted above, in some examples,non-infrared depth sensors, such as passive stereo camera pairs, ornon-identical camera pairs, may be used in projector device 120 in placeof, or in addition to, infrared sensors 146. Such non-infrared depthsensors may be used to determine information about projection surface130 and/or generate a depth map and/or height map of projection surface130. Processing element 304 may build a depth map based on detection bynon-infrared depth sensors of a pattern of light displayed on a surfaceby a light source of projector device 120. Processing element 304 may beeffective to determine the location of various objects in the physicalenvironment within the field of view of image capture device 116 and/orprojector device 120 based on the depth map created by one or morenon-infrared depth sensors.

FIG. 4 depicts four versions of image content, in accordance with anaspect of the present disclosure. Those components of FIGS. 1-3 thathave been described above may not be described again herein for purposesof brevity and clarity.

In the example depicted in FIG. 4, original image 402 may represent aprojection of image data onto a projection surface, such as projectionsurface 130 in FIG. 1. For purposes of the example depicted in FIG. 4,it may be assumed that the projection surface is a relatively uniformbackground in terms of surface geometry and color.

As previously described, image filter unit 140 may be effective toseparate image data into various frequency bands. Thus, image datarepresenting original image 402 may be separated by a high-pass filter,a band-pass filter, and a low-pass filter into high-pass image data,band-pass image data, and low-pass image data. High-pass image data ofthe original image 402 may resemble high-pass content 404 when projectedon a projection surface. Similarly, band-pass image data of the originalimage 402 may resemble band-pass content 406 when projected on aprojection surface. Finally, low-pass image data of the original image402 may resemble low-pass content 408 when projected on a projectionsurface. Although FIG. 4 depicts separation of image content into threefrequency bands, more or fewer frequency bands may be used in accordancewith various other aspects of the present disclosure.

High-pass content 404 may be projected onto a projection surface usingraster projector 142 described above in reference to FIG. 1. As shown inFIG. 4, high-pass content 404 includes relatively high-frequency imagecontent depicting sharp changes in the displayed image. For example,high-pass content 404 shows the outline of the woman depicted in theoriginal image 402 and sharp color transitions in the original image402.

Band-pass content 406 and low-pass content 408 may be projected onto theprojection surface using separate low-frequency projectors 144 describedabove in reference to FIG. 1. Using a raster projector to project thehigh-frequency image data may conserve power relative to using one ormore lamp-based projectors to project the high-frequency image data.Also, raster projectors tend to have smaller form factors relative tolamp-based projectors. Accordingly, the form factor of the projectordevice 120 may be minimized by using a raster projector 142 to projecthigh frequency image data. Additionally, the size of the imagers used inthe low-frequency projectors 144 may be minimized to reduce the size orform factor of projector device 120. In some examples, usingsmaller-sized imagers of low-frequency projectors 144 may result in alower resolution relative to using larger imagers. Accordingly, in somecases, there may be a trade off between image resolution and a size oflow-frequency projectors 144.

As previously described, high-pass content 404, band-pass content 406,and low-pass content 408 may be projected on a projection surface suchthat each frequency band of content overlays one another and aligns withone another. Accordingly, in the example depicted in FIG. 4, thehigh-pass content 404 may be projected by raster projector 142, theband-pass content 406 may be projected overlaying high-pass content 404by a first low-frequency projector 144, and the low-pass content 408 maybe projected overlaying high-pass content 404 and band-pass content 406by a second low-frequency projector 144. The overlapping frequency bandsof image content may be aligned during the alignment calibration process(described above) such that the overlapping image content recreates theoriginal image 402 when viewed on the projection surface.

Although FIG. 4 depicts separation of original image 402 into threefrequency bands, more or fewer frequency bands may be projected inaccordance with various embodiments of the present disclosure. Further,in some examples, augmented image effects, image effect filters, and/orvirtual reality effects may be added to the image content when projectedon the screen. Augmentation of image data using augmented imageenhancement may use edge detection to enhance various portions of theimage. In some examples, the portion of the image to be enhanced may beselected by a user of projector device 120 through an interface. Forexample, a user may select a low-contrast, “glowing aura” augmentedreality effect from an interface configured in communication withprojector device 120 and/or with another computing device over network104 (depicted in FIG. 1). In response, projector device 120 may receiveaugmenting image data representing the augmented reality effect andinstructions for rendering the augmenting image data in conjunction withimage content being projected by projector device 120. For example,augmenting image data may be effective to add highlighting image data tothe low-pass content, to provide a soft “glow” around the depiction ofthe user when such augmenting image data is projected on the projectionsurface 130. Similarly, augmented image data may be added tohigh-frequency image data when sharp contrast is desired for theaugmented image effect. In various examples, “low-contrast” and“high-contrast” augmented image data may be classified by computing adifference between the color and brightness of the particular augmenteddata when rendered on a display and other objects within the field ofview. If the contrast value is calculated to be above a certainthreshold, the augmented data may be considered “high-contrast” and maybe projected using the raster projector 142. Similarly, if the contrastvalue is calculated to be below the threshold, the augmented data may beconsidered “low-contrast” and may be projected using a lamp-basedprojector, such as low-frequency projector 144. In various otherexamples, other quantifiable techniques may be used to categorizeaugmented reality data as low or high contrast. Irrespective of themethod of categorization, augmented data classified as “high contrast”may be projected using raster projector 142 and augmented dataclassified as “low contrast” may be projected using a low frequencyprojector 144. Additionally, in some examples, augmented image data maybe separated into different frequency bands and projected by differentprojectors of projector device 120, as described above. Further, in someexamples, deep learning may be employed to create artistic images out ofimage data. For example, image data representing a photograph of anenvironment may be transformed such that the transformed image dataresembles an oil-painting of the photograph when projected by projectordevice 120.

In other examples, the outline of a subject depicted may be highlightedwith a bright color. Eye-glasses may be projected over the subject'seyes by augmenting the high-pass content 404. Such augmented and/orvirtual image data may be added by processor 148 in accordance withinstructions provided in memory 150 of projector device 120 (depicted inFIG. 1). In other examples, such augmented image data may be providedfrom a content server computing device to projector device 120 overnetwork 104. The content server computing device may likewise provideinstructions for projecting the augmented and/or virtual image data.Such instructions may indicate which projector (e.g., the rasterprojector or a particular frequency-band projector), or whichcombination of projectors, should be used to render the augmented imagedata, as well as a location of the augmented image data in relation tothe original image data. Image effect filters may be applied to imagedata to alter the overall appearance of the image (e.g., to give therendered image a “retro” feel).

FIG. 5 is a flow chart showing an example process for projecting imagedata, in accordance with various aspects of the present disclosure. Theprocess flow 500 of FIG. 5 may be executed by projector device 120and/or by a combination of projector device 120 and a computing deviceconfigured to be in communication with projector device 120, inaccordance with various aspects of the present disclosure. The actionsof process flow 500 may represent a series of instructions comprisingcomputer-readable machine code executable by a processing unit of acomputing device or of projector device 120 or by an image capturedevice, such as image capture device 116. In various examples, thecomputer-readable machine code may be comprised of instructions selectedfrom a native instruction set of the computing device and/or anoperating system of the computing device. Various actions in processflow 500 may be described with reference to elements of FIGS. 1-4.

At action 510 of process flow 500, projector device 120 may identifyimage data, such as image data received from image capture device 116 orfrom a computing device configured in communication with projectordevice 120 over a network.

Processing may continue from action 510 to action 512, “Generate highfrequency image data.” At action 512, the image data identified in step510 may be filtered using a high-pass filter. Accordingly, image datawith a frequency below the cutoff frequency of the high-pass filter maybe attenuated. In the example depicted in FIG. 4, high-pass content 404may be a visual representation of high frequency image data filteredfrom original image data represented by original image 402.

Processing may continue from action 512 to action 514, “Generate lowfrequency image data.” At action 514, the image data identified in step510 may be filtered using a low-pass and/or a band-pass filter. Imagedata with a frequency above the cutoff frequency of a low-pass filtermay be attenuated. Similarly, image data with a frequency outside thepass-band of the band-pass filter may be attenuated. In the exampledepicted in FIG. 4, band-pass content 406 may be a visual representationof low frequency image data filtered from original image datarepresented by original image 402. Similarly, low-pass content 408 maybe another visual representation of low frequency image data filteredfrom original image data represented by original image 402. Band-passimage data may be considered “low frequency” as it is of a lowerfrequency relative to image data generated by a high-pass image filterof projector device 120.

Processing may continue from action 514 to action 516, “Project a firstimage onto a projection surface using a raster scan laser projector.” Ataction 516 a raster scan laser projector, such as raster projector 142may project a first image onto the projection surface. The first imagemay correspond with the high frequency image data generated at action512. In accordance with various embodiments of the present disclosure,projector device 120 may be effective to dynamically adjust the angleand/or distance between raster scan lines depending on the distancebetween the projector device 120 and the portion of the projectionsurface on which the particular scan line is to be projected.Additionally, the raster scan laser projector may emit infrared lightthat may be used to model the projection surface and to determine adistance between the projector device and various portions of theprojection surface.

Processing may continue from action 516 to action 518, “Project a secondimage onto the projection surface overlaying at least a portion of thefirst image using a lamp-based projector.” At action 518, a lamp-basedprojector, such as low-frequency projector 144 may project a secondimage onto the projection surface. The second image may correspond withthe low frequency image data generated at action 514. The lamp-basedprojector may include a telecentric lens used for short-throw projectionapplications. Projector device 120 may be effective to overlay the firstand second images such that the resulting overlapping image is a highquality depiction of the original image data.

FIG. 6 depicts a flow chart showing an example process for projectingimage data, in accordance with various aspects of the presentdisclosure. The process flow 600 of FIG. 6 may be executed by projectordevice 120 and/or by a combination of projector device 120 and acomputing device configured to be in communication with projector device120, in accordance with various aspects of the present disclosure. Theactions of process flow 600 may represent a series of instructionscomprising computer-readable machine code executable by a processingunit of a computing device or of projector device 120 or by an imagecapture device, such as image capture device 116. In various examples,the computer-readable machine code may be comprised of instructionsselected from a native instruction set of the computing device and/or anoperating system of the computing device. Various actions in processflow 600 may be described with reference to elements of FIGS. 1-5.

At action 610 of process flow 600, projector device 120 may identifyimage data, such as image data received from image capture device 116 orfrom a computing device configured in communication with projectordevice 120 over a network.

Processing may continue from action 610 to action 612, “Filter imagedata to generate frequency bands of image data.” For example, imagefilter unit 140 may include one or more high-pass, low-pass, and/orband-pass filters used to filter image data into various frequencyranges of the image data.

Processing may continue from action 612 to action 614, “Project highestfrequency band of image data with raster projector.” At action 614 araster scan laser projector, such as raster projector 142 may project athe highest frequency range (or “band”) of the filtered image datagenerated at action 612 onto the projection surface. In accordance withvarious embodiments of the present disclosure, projector device 120 maybe effective to dynamically adjust the angle and/or distance betweenraster scan lines depending on the distance between the projector device120 and the portion of the projection surface on which the particularscan line is to be projected. Additionally, the raster scan laserprojector may emit infrared light that may be used to model theprojection surface and to determine a distance between the projectordevice and various portions of the projection surface.

Processing may continue from action 614 to action 616, “Project lowfrequency band of image data with lamp-based projector.” At action 616 alamp-based projector, such as low-frequency projector 144 described inreference to FIG. 1, above, may project a different range of filteredimage data apart from the highest frequency image data projected by theraster projector at action 614. For example, at action 616, a lamp-basedprojector may project filtered image data that has passed through alow-pass and/or a band-pass image filter of image filter unit 140.

Processing may continue from action 616 to action 618. At action 618 adetermination may be made whether additional frequency bands are to beprojected. In one example, a processor of projector device 120 (e.g.,processor 148 depicted in FIG. 1) may determine whether additionalfrequency bands are to be projected. In another example, outputs ofvarious high-pass, band-pass, and/or low-pass image filters of imagefilter unit 140 may be coupled to inputs of lamp-based projectors and/orraster projectors of projector device 120. If additional frequency bandsor ranges of filtered image data are to be projected, processing mayreturn from action 618 to action 616.

Conversely, if no frequency bands remain to be projected, processing maycontinue from action 618 to action 620, “Detect light reflected fromprojection surface.” In some examples, infrared sensors may be used todetect infrared light reflected from projection surface 130. In someother examples, a camera, such as camera 152 discussed above inreference to FIG. 1, may be used to detect light reflected fromprojection surface 130. A processing unit of projector device 120 mayuse detected infrared light to model the projection surface 130 and/orto create a depth map of projection surface 130.

Processing may continue from action 620 to action 622 at which adetermination is made whether or not to adjust the image. If adetermination is made, for example by processor 148 or by a processingunit configured in communication with projector device 120, that noimage adjustment is needed, processing may continue from action 622 toaction 626. At action 626, the next image and/or image frame may beprojected in accordance with the process flow described in FIG. 6.

If a determination is made that the image should be adjusted at action622, processing may continue to action 624, “Adjust image based onreflected light.” Projector device 120 may adjust light projected by oneor more raster projectors and/or lamp-based projectors based on thedepth map or other model of projection surface 130. For example,projector device 120 may adjust the dimensions and/or angles ofprojected light to account for irregularities or non-uniformities inprojection surface 130 such that the projected image resembles imagedata stored in a memory of projector device 120. Furthermore, in someexamples, camera 152 may compare color values of the light reflectedfrom projection surface 130 to color values of the currently projectedimage data that are stored in memory (e.g., memory 150 from FIG. 1). Ifprojector device 120 determines that there is a discrepancy between oneor more color values reflected from projection surface 130 and thecorresponding color values stored in memory, projector device 120 mayadjust the color values projected until the color values of reflectedlight match the expected color values stored in memory. Accordingly,projector device 120 may account for different colored projectionsurfaces and for ambient light affecting the color of the projectedimages.

Among other potential benefits, a system in accordance with the presentdisclosure may allow for a small form factor, low power consumption,short throw projector device. Filtering image data into variousfrequency bands and using a raster laser projector to project highfrequency image data may allow for reduced power consumption relative tousing lamp-based projection for high-frequency image data, whileproviding equivalent image quality. Additionally, using a rasterprojector may reduce the size of the form factor of the projector deviceallowing for a smaller and more portable display system. Using infraredand ambient light sensors allows the projector device to detect andcorrect for ambient conditions such as projection surface coloring,geometry, microgeometry, reflectiveness, ambient light, etc. Further,augmented effects may be added to the projected image. Additionally,since the raster laser projector can emit infrared light, no separateinfrared light source need be added in order to detect variousconditions related to the projection surface and/or the surroundingenvironment. In some further examples, infrared sensors may usetime-of-flight technology to determine the distance that various scanlines need to be projected from the raster projector to reach theprojection surface. Angles of emission between raster scan lines may bedynamically adjusted in order to account for divergence of raster scanover the throw of the projector to ensure that a uniform and highquality image is projected onto the projection surface. Variousfrequency bands of the projector device may be overlaid on theprojection surface to recreate the original image. Alignment calibrationand computer vision techniques may be used to align the differentfrequency bands of the image when projected on the projection surface.

Although various systems described herein may be embodied in software orcode executed by general purpose hardware as discussed above, as analternate the same may also be embodied in dedicated hardware or acombination of software/general purpose hardware and dedicated hardware.If embodied in dedicated hardware, each can be implemented as a circuitor state machine that employs any one of or a combination of a number oftechnologies. These technologies may include, but are not limited to,discrete logic circuits having logic gates for implementing variouslogic functions upon an application of one or more data signals,application specific integrated circuits having appropriate logic gates,or other components, etc. Such technologies are generally well known bythose of ordinary skill in the art and consequently, are not describedin detail herein.

The flowcharts and methods described herein show the functionality andoperation of various implementations. If embodied in software, eachblock or step may represent a module, segment, or portion of code thatcomprises program instructions to implement the specified logicalfunction(s). The program instructions may be embodied in the form ofsource code that comprises human-readable statements written in aprogramming language or machine code that comprises numericalinstructions recognizable by a suitable execution system such as aprocessing component in a computer system. If embodied in hardware, eachblock may represent a circuit or a number of interconnected circuits toimplement the specified logical function(s).

Although the flowcharts and methods described herein may describe aspecific order of execution, it is understood that the order ofexecution may differ from that which is described. For example, theorder of execution of two or more blocks or steps may be scrambledrelative to the order described. Also, two or more blocks or steps maybe executed concurrently or with partial concurrence. Further, in someembodiments, one or more of the blocks or steps may be skipped oromitted. It is understood that all such variations are within the scopeof the present disclosure.

Also, any logic or application described herein that comprises softwareor code can be embodied in any non-transitory computer-readable mediumor memory for use by or in connection with an instruction executionsystem such as a processing component in a computer system. In thissense, the logic may comprise, for example, statements includinginstructions and declarations that can be fetched from thecomputer-readable medium and executed by the instruction executionsystem. In the context of the present disclosure, a “computer-readablemedium” can be any medium that can contain, store, or maintain the logicor application described herein for use by or in connection with theinstruction execution system. The computer-readable medium can compriseany one of many physical media such as magnetic, optical, orsemiconductor media. More specific examples of a suitablecomputer-readable media include, but are not limited to, magnetic tapes,magnetic floppy diskettes, magnetic hard drives, memory cards,solid-state drives, USB flash drives, or optical discs. Also, thecomputer-readable medium may be a random access memory (RAM) including,for example, static random access memory (SRAM) and dynamic randomaccess memory (DRAM), or magnetic random access memory (MRAM). Inaddition, the computer-readable medium may be a read-only memory (ROM),a programmable read-only memory (PROM), an erasable programmableread-only memory (EPROM), an electrically erasable programmableread-only memory (EEPROM), or other type of memory device.

It should be emphasized that the above-described embodiments of thepresent disclosure are merely possible examples of implementations setforth for a clear understanding of the principles of the disclosure.Many variations and modifications may be made to the above-describedexample(s) without departing substantially from the spirit andprinciples of the disclosure. All such modifications and variations areintended to be included herein within the scope of this disclosure andprotected by the following claims.

What is claimed is:
 1. A short-throw display system, comprising: atleast one processor effective to: receive image data; filter the imagedata with a high-pass filter to produce first frequency range image datacorresponding to a first frequency range; filter the image data with alow-pass filter to produce second frequency range image datacorresponding to a second frequency range, wherein the second frequencyrange image data comprises lower frequencies relative to frequencies ofthe first frequency range image data; a laser projector effective toreceive the first frequency range image data and to project a firstimage onto a projection surface with laser light in a raster scanningpattern; and a lamp-based projector effective to receive the secondfrequency range image data and to project a second image onto theprojection surface at least partially aligned with the first image;wherein the short-throw display system comprises a throw ratio of 0.4:1or greater.
 2. The short-throw display system of claim 1, furthercomprising: an infrared depth sensor effective to detect infrared laserlight reflected from the projection surface; and wherein the at leastone processor is effective to generate a depth map of the projectionsurface based at least in part on the infrared laser light, wherein thedepth map indicates a non-uniform element of the projection surface, andthe laser projector is effective to adjust the first image to compensatefor the non-uniform element of the projection surface.
 3. Theshort-throw display system of claim 1, wherein the at least oneprocessor is effective to filter the image data with a band-pass filterto produce third frequency range image data corresponding to a thirdfrequency range, wherein the third frequency range comprises frequenciesbetween the first frequency range and the second frequency range, thesystem further comprising: a second lamp-based projector effective toreceive the third frequency range image data and to project a thirdimage onto the projection surface at least partially aligned with thefirst image.
 4. A display system, comprising: an image filter uniteffective to filter image data into at least first frequency range imagedata corresponding to a first frequency range and second frequency rangeimage data corresponding to a second frequency range, wherein the firstfrequency range comprises higher frequencies relative to the secondfrequency range; a laser projector effective to receive the firstfrequency range image data and to project a first image onto aprojection surface using a raster-based technique, wherein the firstimage corresponds to the first frequency range image data; a lamp-basedprojector effective to receive the second frequency range image data andto project a second image onto the projection surface, wherein thesecond image at least partially aligns with the first image andcorresponds to the second frequency range image data; and wherein thedisplay system comprises a throw ratio of 0.4:1 or greater.
 5. Thedisplay system of claim 4, further comprising: an infrared sensor; andat least one processor programmed to: cause the laser projector to emitinfrared light toward the projection surface; cause the infrared sensorto detect a portion of the infrared light reflected from the projectionsurface; and generate a depth map of the projection surface based atleast in part on the portion of the infrared light reflected from theprojection surface.
 6. The display system of claim 4, wherein a firstimage filter of the image filter unit comprises a high-pass filter and asecond image filter of the image filter unit comprises a low-passfilter.
 7. The display system of claim 6, wherein a third image filterof the image filter unit is effective to filter image data into thirdfrequency range image data corresponding to a third frequency range,wherein the third frequency range comprises frequencies between thefirst frequency range and the second frequency range, the system furthercomprising: a third projector effective to receive the third frequencyrange image data and to project a third image onto the projectionsurface, wherein the third image at least partially aligns with thefirst image.
 8. The display system of claim 4, wherein the lamp-basedprojector comprises a telecentric lens with a first field of view thatat least partially overlaps with a second field of view of the laserprojector.
 9. The display system of claim 4, further comprising: acamera effective to detect light reflected from the projection surface;and at least one processor effective to: determine a first color valueof the light reflected from the projection surface; compare the firstcolor value of the light to a second color value of image data stored ina memory; and adjust a third color value of the first image based atleast in part on a difference between the first color value and thesecond color value.
 10. The display system of claim 4, wherein thelamp-based projector comprises a lamp and is configured to project thesecond image using light emitted from the lamp.
 11. The display systemof claim 4, further comprising: at least one processor; an infraredsensor; and a scanning mirror; wherein the at least one processor iseffective to: determine a first distance between the laser projector anda first position on the projection surface where a first raster scanline is projected, where the first distance is determined based at leastin part on first light reflected from the first position on theprojection surface and detected by the infrared sensor; determine asecond distance between the laser projector and a second position on theprojection surface where a second raster scan line is projected, wherethe second distance is determined based at least in part on second lightreflected from the second position on the projection surface anddetected by the infrared sensor; determine that the second distance isgreater than the first distance; and control the scanning mirror todecrease a projection angle between the projection of the first rasterscan line and the second raster scan line.
 12. The display system ofclaim 4, wherein: at least one of the laser projector and the lamp-basedprojector is further effective to receive second image data, wherein thesecond image data comprises an image effect configured to augment atleast a portion of one or more of the first image or the second image;and at least one of the laser projector and the lamp-based projector iseffective to project the second image data in conjunction with the firstimage and the second image.
 13. A method comprising: identifying imagedata; generating first image data of a first frequency range from theimage data; generating second image data of a second frequency rangefrom the image data, wherein the first frequency range is higher thanthe second frequency range; projecting a first image onto a projectionsurface with a laser projector using a raster-based technique, whereinthe first image corresponds to the first image data; and projecting asecond image onto the projection surface aligned with least a portion ofthe first image, wherein projecting the second image onto the projectionsurface is performed using a lamp-based projector, and wherein thesecond image corresponds to the second image data, wherein the laserprojector and the lamp-based projector each comprise a throw ratio of0.4:1 or greater.
 14. The method of claim 13, further comprising:emitting light toward the projection surface; detecting a portion of thelight reflected from the projection surface with a depth sensor; andgenerating a depth map of the projection surface based at least in parton the portion of the light reflected from the projection surface. 15.The method of claim 13, further comprising: filtering the image datawith a high-pass filter to generate the first image data of the firstfrequency range; and filtering the image data with one or more of alow-pass filter or a band-pass filter to generate the second image data.16. The method of claim 15, further comprising: filtering the image datawith a second band-pass filter to generate third image data; andprojecting a third image onto the projection surface, wherein theprojected third image is at least partially aligned with the first imageand the second image, wherein the projected third image corresponds tothe third image data.
 17. The method of claim 13, further comprising:detecting light reflected from the projection surface with a lightsensor; determining a first color value of the light reflected from theprojection surface; comparing the first color value of the light to asecond color value of the image data stored in a memory; and adjusting athird color value of the first image or the second image based at leastin part on a difference between the first color value and the secondcolor value.
 18. The method of claim 13, further comprising: determininga first distance between the laser projector and a first position on theprojection surface where a first raster scan line is projected, wherethe first distance is determined based at least in part on firstinfrared light reflected from the first position on the projectionsurface and detected by an infrared sensor; determining a seconddistance between the laser projector and a second position on theprojection surface where a second raster scan line is projected, wherethe second distance is determined based at least in part on secondinfrared light reflected from the second position on the projectionsurface and detected by the infrared sensor; determining that the seconddistance is greater than the first distance; and adjusting a projectionangle of the laser projector to maintain a substantially uniformdistance between consecutive raster scan lines projected on theprojection surface.
 19. The method of claim 13, further comprising:identifying second image data, wherein the second image data comprisesan image effect configured to augment at least a portion of one or moreof the first image or the second image; receiving an instruction to addthe second image data to the image data; and projecting an augmentedeffect corresponding to the second image data in conjunction with thefirst image and the second image.